Hardware vs. Simulation: Bridging the Gap between Efficiency and Flexibility for Computer Engineering Education and Research

Computer architecture and systems are fundamental areas in computer engineering. One of the most critical challenges facing computer engineering education and research is the characterization of the underlying complex computer systems. Traditionally, this can be done via either real hardware measurement or software simulation, each having its own pros and cons. Running workloads and measuring performance characteristics on the real hardware is efficient and accurate: this is the more direct approach of obtaining system measurements, and the results are guaranteed to be accurate. However, the hardware platform can only provide limited insights into its design details and little flexibility for extension. On the other hand, various simulators have been developed to model different computer architectures, facilitating an expedited design process. Experiments using simulations are highly flexible and effective in quickly testing various combinations of design parameters. In order to bridge the gap and exploit benefits from both approaches, we investigate a number of innovative mechanisms combining both hardware and simulation. First, hardware performance counters implemented in current CPUs can be used to monitor live running systems’ activity factors, which can be used together with analytical simulations to generate fast and accurate estimations of interesting metrics, e.g. system power consumption, among many others. Second, the so-called “co-simulation” scheme enables the simulator to run directly on the same platform that it simulates and switches between the detailed simulation mode and native system mode at any time. As a result, difficult executions can be quickly performed on the real system. Third, a few representative hardware systems can be used for performance and activity measurements, and then serve as the reference machines based upon which the characteristics of a new hypothetical machine can be easily obtained via reasonable scaling.